Bring the wire format and API to parity with TypeScript SDK 3.0.1 - #84
Merged
Conversation
PR #83 mirrors the TypeScript SDK's PR #146. The TypeScript side then took a review round, #147, which changed the container formats and the verification semantics and shipped as 3.0.0. This is that round. The two SDKs already agreed on everything sent to the aggregator — CertificationData bytes are identical down to the golden vectors, as are the transaction encodings, the inclusion proof and the leaf value. What diverged is the token: this SDK cannot read a token 3.0.0 produces, and 3.0.0 cannot read one produced here. Wire: - Token.VERSION 1 -> 2. Every structure it embeds changed shape, so a token written by the other version now fails the version check rather than dying further down on a CBOR array-length error that never mentions versioning. - Certified mint and transfer arrays lose their middle element, 3 -> 2. The service records the leaf's creation time on the record and serves that same value for every proof of the leaf, so the copy stored beside the proof could never legitimately differ from it; it cost a wire element and a consistency check that could only ever agree. Verification: - The rule reads the reference time from the proof instead of being handed it, so REFERENCE_TIME_MISMATCH has nothing left to compare and is gone. - A leaf claiming to postdate the round that certified it is rejected (REFERENCE_TIME_AFTER_ROUND). Consensus signs the round timestamp, so the pairing cannot occur legitimately. Read the comment on that check before relying on it: the bound is one-sided and does not stop back-dating, which is the direction an attacker wants. - A proof reporting no leaf at all is the only answer treated as "not certified yet". A partially present proof now names what is missing instead of reading as pending and leaving a caller polling to its own deadline. - Binding a transaction to a proof for an uncertified state reports INCLUSION_CERTIFICATE_MISSING again; a guard in both factories was reporting a missing reference time, which no retry path recognises. - expiresAt is validated where it is accepted rather than failing later inside CBOR encoding. Both deadline comparisons are unsigned. This has no counterpart in the TypeScript SDK, whose bigint does not wrap: here a CBOR unsigned integer at or above 2^63 arrives as a negative long (CborDeserializer.CborUnsignedLong.asLong says so), and a signed comparison would read such a reference time as earlier than every deadline and wave an expired request through while the leaf value, computed from the same bits, still verified. Fixture certificates now certify a round whose clock matches the leaf. They defaulted to a timestamp of zero while leaves claimed 1755000000 — a pairing no aggregator can produce, and one the new bound rejects.
Interop. Each SDK builds a token from entirely fixed inputs — keys, salt, token type, state mask, deadline and the fake aggregator's round clock, with RFC 6979 signing on both sides — commits it, and decodes and fully verifies the other's. This is the test that would have caught the divergence this branch fixes. The CrossSdkEncodingTest vectors both SDKs already carry pin CertificationData, and those bytes never moved: with Token.VERSION reverted to 1, CrossSdkEncodingTest passes 2/2 while both interop tests fail. Only carrying a real token across the language boundary exercises Token, the certified transactions inside it, and the verification semantics that read them. The two SDKs' UnicityCertificate test fixtures differ in three padding fields, so a shared token hex vector is not possible. It is not needed: each side reads the producer's certificate and trust base out of the fixture bundle. TestAggregatorClient gains a reference-time setter, so a generated vector is byte-reproducible rather than dependent on when it was generated. Integration. AggregatorStack starts the compose stack from the TypeScript SDK's own file — BFT root node, mongodb, redis and a pinned aggregator build — waits for consensus to certify a round rather than for the healthcheck, and tears it down after. RequestDeadlineIntegrationTest mirrors the TypeScript cases: the exclusive deadline at submission, the service-assigned branch, what the leaf carries back, and the round-timestamp relation. Tagged `integration`, so the existing integrationTest task picks it up and the ordinary test task keeps excluding it. No build change was needed. These integration tests have NOT been observed to pass. They compile and are correctly excluded from `test`, but the machine they were written on runs Docker 29, whose minimum API version docker-java does not meet, so the suite could not be executed end to end. Run `./gradlew integrationTest` on a normal Docker host before trusting them.
The pinned 1.19.8 could not reach a current Docker daemon at all. docker-java 3.4.x negotiates API 1.32; Docker 29 requires 1.44 and refuses the connection, so every Testcontainers-based test failed before starting a container — including, until now, the integration suite added in the previous commit. Testcontainers 2.0.5 carries docker-java 3.7.1 and connects. Two things fall out of the 2.x move: - junit-jupiter and mongodb are dropped. They were declared but never used — nothing in this repo imports @testcontainers, @container or MongoDBContainer — and 2.x does not publish them. Only the core artifact is needed, for ComposeContainer and Wait. - 2.x removed containerised compose, so ComposeContainer shells out to the docker CLI. That is present on any CI runner and on a developer machine; it was not present in the JDK container this was written in, which is what made the suite look unrunnable rather than merely unrun. With that, RequestDeadlineIntegrationTest passes against a real aggregator: 8 tests, no skips, about 16 seconds once the stack is up. The stack tears down after and leaves no containers and no generated genesis behind.
MastaP
marked this pull request as ready for review
August 27, 2026 10:03
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 47260daae6
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
… vector The first version of this carried committed token vectors and a matching generator in the TypeScript repo, so a cross-SDK check needed a change in both repos and a blob copied between them. It does not. The npm package ships only lib/, and everything needed to mint a token is in it — the fake aggregator that the vector generator leaned on is test code and is not published. Since this suite already starts a real aggregator, the TypeScript SDK can mint against that one instead. So: a node container runs the published @unicitylabs/state-transition-sdk@3.0.0 against the aggregator this suite started, mints and transfers a token, verifies it with its own SDK, and prints it. Java decodes and verifies the result. Better than the vector it replaces in three ways. It exercises the artifact a consumer installs rather than the other repo's source tree. The token is certified by a real aggregator, so real signatures and certificates rather than fake-aggregator-shaped ones. And nothing is committed, so there is no blob to go stale and no question of who regenerates it — which also removes the need for the deterministic-fixture machinery, and for the TypeScript-side PR entirely. Two things worth knowing about the wiring: - The node step shells out to the docker CLI rather than using a Testcontainers GenericContainer. Testcontainers already requires that CLI for ComposeContainer so it adds no dependency, and a one-shot container that fails reports its own output instead of "did not start correctly" with empty logs. - The container joins the stack's network and addresses the aggregator by service name, which needs no published port and no host-gateway assumption. The network name is read off the running container; deriving it from the compose project name produced a name that did not exist.
The comments added in this branch ran to over half the lines in src/main, and the long ones buried what they were explaining: twelve lines of prose above a three-line comparison. Trimmed to the parts that are not evident from the code — why the comparisons are unsigned, what the pending status means, and that the round bound is one-sided — with the back-dating argument left to aggregator-go#186 rather than restated in full at the call site.
InclusionProof carried both answers the aggregator can give — a certified leaf, and the absence of one — so every field was nullable and every consumer had to re-establish which case it held. That produced a status taxonomy describing states a proof should never have been able to be in, guards in both certified transaction decoders, Optional round-trips on the reference time, and four absence branches at the top of the verification rule before any verifying began. The absence belongs to the response, not to the proof: - InclusionProof requires certificationData, referenceTime and inclusionCertificate. getReferenceTime returns long, getCertificationData returns the data. A value of this type describes a certified leaf; there is no other thing it can be. - InclusionProofResponse carries a nullable proof plus the certificate the answer was served against, and is the type that can say "not certified yet". - The wire form expresses both, so decoding it stays where the layout lives: decodeOrAbsent returns the proof or null and rejects any partial combination, and fromCbor refuses anything but a leaf. MISSING_CERTIFICATION_DATA, MISSING_REFERENCE_TIME, INCOMPLETE_INCLUSION_PROOF and INCLUSION_CERTIFICATE_MISSING are gone — none of them can occur. The poll loop branches on the response having no proof rather than on a status meaning the same thing, and its switch collapses to an if. The wire bytes do not change. The interop test proves it: a token minted by the published TypeScript SDK, which does not have this split, still decodes and verifies here unchanged.
Three changes the TypeScript side made after the parity work in this branch was written, applied here so the two SDKs match in shape and not only on the wire. A transfer decodes from its source, not from the whole token. TransferTransaction.fromCbor and CertifiedTransferTransaction.fromCbor take the state being spent and the lock script over it rather than a Token; both are checked against the certification data during verification, so a wrong value fails there. That removes the self-reference the token requirement forced: Token.fromCbor used to construct a Token over a mutable list and add to it while decoding, so getLatestTransaction would advance and each transfer could read its source back off a half-built token. The chain is derived where it belongs now. The response cannot contradict its own proof. InclusionProofResponse had a public three-argument constructor, so a caller could supply a certificate differing from the one inside the proof — and toCbor serialises the proof's, so the field was not preserved across a round trip. The constructor is private and there are two named factories: certified() reads the certificate off the proof, notCertified() takes one because there is no proof to read it from. The wire's two shapes live in the response. decodeOrAbsent and encodeNoCertifiedLeaf were on InclusionProof, which is the type that cannot represent an absent leaf — the same leak the split was meant to close, one level up. InclusionProofResponse decodes the tagged structure itself, decides certified from not, and builds the InclusionProof from the parts. InclusionProof.fromCbor is self-contained and rejects anything but a leaf. The interop test now pins the published 3.0.1 rather than 3.0.0, so what it proves is agreement with the current release.
This SDK is the counterpart of state-transition-sdk-js 3.0.1 and shares its wire formats, so the version lines are brought together. There is no 2.x. A release still supplies its own version: release.yml is dispatched with one and passes it as -Pversion.
hasProperty("version") is always true, because Gradle defines version as a
project property. The else branch had therefore never run: the fallback sat at
1.1-SNAPSHOT through the whole 1.2 to 1.4.2 series without effect, and a build
without -Pversion produced artifacts with no version in the name at all.
Checking for the "unspecified" that property holds when -Pversion was not
passed makes the fallback do what it looks like it does. A local build now
reports 3.0-SNAPSHOT and names its jars accordingly; a release passing
-Pversion is unchanged.
Readiness can time out and the service lookups can throw, all on a stack that is already running. Nothing holds the environment until the constructor runs, so close() could never be reached: a failed startup left the whole stack and its genesis behind, and the next run would then try to delete a directory that running containers had mounted. Cleanup failures are attached to the original exception rather than replacing it, so a teardown problem cannot hide why startup failed.
MastaP
added a commit
to unicity-sphere/sphere-sdk
that referenced
this pull request
Aug 28, 2026
Replaces the manual `make docker-run-clean` + AGGREGATOR_URL/AGGREGATOR_TRUSTBASE setup with a stack the suite starts itself. `npm run test:aggregator` is now the whole instruction: cold start to green in about 1m45s, against nothing. The compose is copied from state-transition-sdk-js `tests/integration/docker/docker-compose.yml`, which the Java SDK mirrors as well (unicitynetwork/state-transition-sdk-java#84). Keeping the three identical is the point — if each repo exercised a different service build, "passes against a real aggregator" would mean something different in each one. It pins the prebuilt ghcr.io/unicitynetwork/aggregator-go image, so no local rocksdb build: that alone was the ~10 minutes the manual path cost on a cold machine. Two things the harness gets right that a naive port would not: - It waits for consensus to CERTIFY A ROUND, not for the container to report healthy. Until the service has a reference time it answers every certification request SERVICE_NOT_READY, so a health-gated wait races startup. - It wipes the bind-mounted genesis before AND after. Genesis survives a container teardown while the mongodb and redis volumes do not, so a reused directory pairs a chain that remembers nothing with a root node that remembers everything. The suite always starts its own stack rather than accepting a URL. Pointing it at someone else's service is what test:e2e is for, and would mean a green run proved nothing about the compose file this suite exists to exercise.
MastaP
added a commit
to unicity-sphere/sphere-sdk
that referenced
this pull request
Aug 28, 2026
…e) — ships as 0.15.0 (#761) * feat!: state-transition-sdk 3.0.1 — request deadlines and reference time v3 threads one new concept through the protocol: every transaction carries `expiresAt`, an exclusive request deadline, and every inclusion proof carries the `referenceTime` of the round that certified it. The sparse-Merkle leaf is now H(transactionHash, referenceTime) rather than the bare hash, and every wire version underneath moved — nothing 2.x wrote decodes, and nothing 2.x writes is accepted. Sphere sets no deadline anywhere. Not for determinism — a deadline persisted on the durable intent would rebuild byte-identically — but because this is a browser wallet with an untrusted clock, and because an absolute deadline is unrecoverable across downtime longer than the window: every resume would rebuild an already-expired transaction and the intent would sit open forever with its sources reserved. Omitted means the service assigns one from consensus time and does not record it. That policy is load-bearing rather than cosmetic. `expiresAt` is committed by the transaction hash but is NOT part of the StateId, so two attempts that disagree about it address the same leaf with different hashes — and the verification rule compares the hash first, so the disagreement surfaces as TRANSACTION_HASH_MISMATCH, i.e. as a foreign spend. A clock-derived deadline would make every crash-resume abort an intent whose spend is already on chain. expires-at.test.ts pins it, including a 24-hour clock jump between attempts. REQUEST_EXPIRED and SERVICE_NOT_READY stay out of CLEAN_REJECT_STATUSES despite their names: each reports only that THIS submit was not admitted, never that no earlier attempt certified. SERVICE_NOT_READY is instead retried at the submit call site — it is a 503 in a 200 body, and left alone it surfaced CERTIFICATION_UNCONFIRMED for a booting gateway. 3.0.1 also reshaped the proof types: `InclusionProofResponse.inclusionProof` is now nullable and null IS "not certified yet", so `isSpent` and the split pre-flight read the response instead of digging for certification data. Closes #760 * test: probe the two invariants the 3.x migration introduced The deadline policy and the KV generation rename are both money-critical and both invisible in a type signature, so they get probes rather than only tests. `engine-sets-clock-derived-deadline` reproduces the failure the policy exists to prevent: a deadline read off the wall clock makes a crash-resumed rebuild address the same leaf with a different transaction hash, which the rule reports as TRANSACTION_HASH_MISMATCH and the engine maps to TransferConflictError — so the wallet aborts an intent whose spend is already on chain, demotes the source, and re-sends the amount from other sources. `kv-generation-not-renamed` puts the prefix back on `pv2:`, which is enough for a surviving sync-epoch latch to make a backend reset look like a server restore and re-PUT dead intents into the fresh backend. Also re-anchors submit-429-not-retried, whose find text moved with the SERVICE_NOT_READY retry, and corrects three comments that still named the old `pv2:` prefix or the removed paymentsV2 alias. * docs: document the 3.x flag day, and correct what the bump exposed Rewrites CLAUDE.md, the changelog, the design docs and the consumer docs for the state-transition-sdk 3.0.1 migration: what v3 is, why sphere sets no request deadline, why the two new time-dependent statuses are not clean rejects, the TokenBlob removal, the split-checkpoint version bump, the scoped-KV generation rename, and the removal of the paymentsV2 alias — including the behavioural difference consumers will otherwise meet in production, that `sphere.payments` throws where the alias returned null. Several corrections are to statements that were already false before this branch, found while verifying the ones being written: - The canonical Quick Start would have thrown INVALID_CONFIG. `Sphere.init` resolves the payments composition from its OWN `network` and compares it to `walletApi.network` as a string, but the example passed `network` only to createBrowserProviders (where it lands inside the oracle argument, not at bundle top level) and used the 'testnet' alias against 'testnet2'. README called `network` on Sphere.init "optional/informational"; it is required and all three must agree. - `history:updated` was documented as an empty payload in three files. It has carried a HistoryEntry since c4fb918. - docs/PAYMENTS-V2-DESIGN.md §5.5 justified never retrying REQUEST_EXPIRED with "a re-submit is a different transaction hash" — false here precisely because sphere sets no deadline, so the rebuild is byte-identical. The real reason is that reference time only advances. - docs/MIGRATION-PAYMENTS-V2.md §5's mixed-version compatibility matrix claimed interop that no longer exists in either direction. - LEGACY-INVENTORY records the 3.x removals. Those entries were correctly refuted as unremovable when written; the bump changed the facts, not the reasoning, and the pass-through arm the refutation rested on survives as the plain Token.fromCBOR in deriveDeliveryKeys. * docs: cut the padding this bump did not need The docs pass ran wide and over-wrote. Removed, without losing a fact: - The README release-note section. It duplicated the changelog, went stale on the next release, and introduced a section convention this repo does not have. The paymentsV2 migration guidance stays but is a third of its length; the deeper version lives in INTEGRATION.md, which is where an upgrader looks. - The expiresAt rationale had four copies. It now has two — CLAUDE.md, because that file is the standing context, and the CHANGELOG, because that is the release record. INTEGRATION.md points at them instead of restating. - CLAUDE.md's enumeration of the changed SDK signatures. Exact shapes are in the types; the file needs the POLICY, not a second copy of the changelog. - CONNECT.md repeated "getBalance and getAssets are the same call" three times. - The changelog's API paragraph and host-shape section, tightened. Kept deliberately: docs/VERIFICATION-WORKERS.md grew because the entry script it documents was broken before this branch — the worker base has required two verifiers since 2.1.0, so the published script did not compile. PAYMENTS-V2-DESIGN and MIGRATION-PAYMENTS-V2 grew because they carried false statements about determinism inputs and cross-version interop. Docs diff: 1050 added -> 928. * chore: release v0.15.0-dev.1 * fix(connect): keep compat events alive while the facade is torn down Codex P2 on #761, and it is a regression this branch introduced. `stopPaymentsV2Inner` clears `_paymentsV2Active` BEFORE awaiting `PaymentsFacade.stop()` (core/Sphere.ts) — deliberately, so no new operation can start against a stopping facade. But stop() is exactly where in-flight ops settle and emit, so a `payment_request:updated` can arrive while the public `sphere.payments` getter throws NOT_INITIALIZED. Sphere's `emitEvent` catches a throwing handler and only logs it. So the compat adapter's unguarded read did not surface an error to anyone — the dApp simply never received the terminal paid/rejected/expired event. The `sphere.paymentsV2` alias this replaced was optional-chained, so the same window produced the degraded id + status payload instead. That is the null-vs-throws difference the release notes warn consumers about, reintroduced inside our own call sites. `paymentsOrNull()` restores the previous semantics at the two EVENT call sites. The query-router reads stay unguarded on purpose: a throw there becomes a typed ConnectError the dApp can see and retry, which the lock suite already pins. Two tests, both failing before the fix. * test: prove the wire against a real aggregator-go v3, not just against ourselves Every other suite that exercises the chain runs on TestAggregatorClient, which orchestrates the SDK's own SMT, CertificationData and verification rule. That proves this client is self-consistent; it cannot prove the client and the SERVICE agree, because both sides were built from the same written spec and a spec can be read two ways. The integration suites prove less about the wire still — they swap a fake engine in through setEngine, so they pin facade orchestration and would pass with the CBOR wrong. That was the gap worth closing before a flag day with no straddle window, and it does not need testnet: aggregator-go ships a compose that stands up a real root + aggregator locally and generates its own trust base. tests/aggregator/ runs the real engine against it — mint, transfer with the RECIPIENT's own engine verifying, split with value conserved, and a same-transferId re-call recovering the byte-identical token. That last one is E.1 determinism against a real service, which is exactly what the no-deadline policy protects. `verify()` passing is the assertion no fake can make: the leaf this client computes — H(transactionHash, referenceTime) since 3.x, where 2.x used the bare hash — reproduces the leaf the Go service actually inserted, and the BFT certificate chains to the service's own trust base. Checked it is not vacuous: a well-formed but wrong root key fails INVALID_TRUSTBASE. Opt-in via AGGREGATOR_URL + AGGREGATOR_TRUSTBASE, like test:relay — skipped, not failed, when unset, so test:run is untouched. * test(aggregator): stand the stack up with Testcontainers, not by hand Replaces the manual `make docker-run-clean` + AGGREGATOR_URL/AGGREGATOR_TRUSTBASE setup with a stack the suite starts itself. `npm run test:aggregator` is now the whole instruction: cold start to green in about 1m45s, against nothing. The compose is copied from state-transition-sdk-js `tests/integration/docker/docker-compose.yml`, which the Java SDK mirrors as well (unicitynetwork/state-transition-sdk-java#84). Keeping the three identical is the point — if each repo exercised a different service build, "passes against a real aggregator" would mean something different in each one. It pins the prebuilt ghcr.io/unicitynetwork/aggregator-go image, so no local rocksdb build: that alone was the ~10 minutes the manual path cost on a cold machine. Two things the harness gets right that a naive port would not: - It waits for consensus to CERTIFY A ROUND, not for the container to report healthy. Until the service has a reference time it answers every certification request SERVICE_NOT_READY, so a health-gated wait races startup. - It wipes the bind-mounted genesis before AND after. Genesis survives a container teardown while the mongodb and redis volumes do not, so a reused directory pairs a chain that remembers nothing with a root node that remembers everything. The suite always starts its own stack rather than accepting a URL. Pointing it at someone else's service is what test:e2e is for, and would mean a green run proved nothing about the compose file this suite exists to exercise. * fix(test): keep the docker-backed aggregator suite out of test:run The default vitest config globs tests/**, and dropping the env gate meant the Testcontainers suite ran inside `npm run test:run` — quietly making the default suite require Docker and adding ~2 minutes to it. Excluded alongside e2e and relay, which are out for the same reason. Caught by the file count: 117 -> 118. --------- Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Follow-up to #83, targeting
service-timeso it can be folded in before that PR merges.#83 mirrors the TypeScript SDK's #146. The TypeScript side then took two review rounds — #147, which shipped as 3.0.0, and #150/#151, which shipped as 3.0.1. This brings the Java SDK to the 3.0.1 shape, so the two match in structure and not only on the wire.
The interop test pins the published 3.0.1, so what it proves is agreement with the current release.
Beyond the 3.0.0 parity work
TransferTransaction.fromCborandCertifiedTransferTransaction.fromCbortake the state being spent and the lock script over it; both are checked against the certification data during verification, so a wrong value fails there. That removed the self-reference the token requirement forced —Token.fromCborused to construct aTokenover a mutable list and add to it while decoding, sogetLatestTransactionwould advance and each transfer could read its source back off a half-built token.toCborserialises the proof's — so the field was not preserved across a round trip. Private constructor, pluscertified()andnotCertified().decodeOrAbsentandencodeNoCertifiedLeafwere onInclusionProof, the type that cannot represent an absent leaf.InclusionProofResponsedecodes the tagged structure itself and builds the proof from the parts;InclusionProof.fromCboris self-contained and rejects anything but a leaf.What already agreed, and what did not
The two SDKs agree on everything sent to the aggregator. I verified this rather than assuming it — the
CertificationDatagolden vectors are byte-identical in both repos, for the explicit and the absent deadline:MintTransactionv2 (8 fields),TransferTransactionv2 (5 fields),InclusionProof(5 elements, version 1, all-or-none leaf invariant) and the leaf valueSHA256(CBOR([txhash, referenceTime]))all matched line for line too.What diverged is the token. This SDK cannot read a token 3.0.0 produces, and 3.0.0 cannot read one produced here.
Wire changes
Token.VERSIONCertifiedMintTransactionCertifiedTransferTransactionThe middle element was a copy of the leaf's reference time, stored beside a proof that already carries it. The service records that time on the record and serves the same value for every proof of the leaf, so the copy could never legitimately differ — it cost a wire element and a consistency check that could only ever agree.
Verification changes
InclusionProofused to carry both answers the aggregator can give — a certified leaf, and the absence of one — so every field was nullable and every consumer re-established which case it held. That is now split:The wire form expresses both shapes, so decoding it stays where the layout lives:
decodeOrAbsentreturns the proof or null and rejects any partial combination;fromCborrefuses anything but a leaf.What falls out: the whole status taxonomy for absent fields is gone —
MISSING_CERTIFICATION_DATA,MISSING_REFERENCE_TIME,INCOMPLETE_INCLUSION_PROOF,INCLUSION_CERTIFICATE_MISSING— because none of them can occur. Both certified-transaction decoders drop their guards. The poll loop branches on the response having no proof rather than on a status that meant the same thing, and itsswitchcollapses to anif. There is no null handling left anywhere in the verification path.The wire bytes do not change, and the interop test proves it rather than asserting it: a token minted by the published TypeScript SDK, which does not have this split, still decodes and verifies here unchanged.
The remaining behaviour changes:
REFERENCE_TIME_AFTER_ROUND). Read the comment on that check before relying on it — the bound is one-sided and does not stop back-dating. Filed as Inclusion proofs do not let a verifier detect a back-dated reference time, so expiresAt is unenforceable against a dishonest aggregator aggregator-go#186.expiresAtis validated where it is accepted rather than failing later inside CBOR encoding.One thing with no TypeScript counterpart
Both deadline comparisons are unsigned. TypeScript's
bigintdoes not wrap; a Javalongdoes.CborDeserializer.CborUnsignedLong.asLongsays so in its own javadoc: a CBOR unsigned integer at or above 2^63 comes back as a negative long. A signed comparison would read such a reference time as earlier than every deadline and wave an expired request straight through — while the leaf value, computed from the same bits, still verified. BothREQUEST_EXPIREDand the round bound now useLong.compareUnsigned.This was found by review, not by a failing test, and it would have shipped otherwise.
Interop against the published TypeScript SDK
A node container runs
@unicitylabs/state-transition-sdk@3.0.0from npm against the aggregator this suite already starts, mints and transfers a token, verifies it with its own SDK, and prints it. Java decodes and verifies the result.The argument for it, concretely: revert
Token.VERSIONto 1 — the state this branch starts from — andCrossSdkEncodingTestpasses 2/2 while the interop test fails. The existing golden vectors are structurally incapable of seeing a container divergence, becauseCertificationDatabytes never move. That is exactly how this divergence survived until now.Three properties worth noting:
The node step shells out to the docker CLI rather than using a Testcontainers
GenericContainer. Testcontainers already requires that CLI forComposeContainer, so it adds no dependency, and a one-shot container that fails reports its own output rather than "did not start correctly" with empty logs. The container joins the stack's network and addresses the aggregator by service name, so it needs no published port and no host-gateway assumption.Integration tests
AggregatorStackstarts the compose stack from the TypeScript SDK's own file — BFT root node, mongodb, redis, pinned aggregator build — waits for consensus to certify a round rather than for the healthcheck, and tears it down.RequestDeadlineIntegrationTestmirrors the TypeScript cases.Tagged
integration, so the existingintegrationTesttask picks it up andtestkeeps excluding it. No build change was needed for that.No containers and no generated genesis left behind afterwards.
Testcontainers 1.19.8 → 2.0.5
This is a fix, not housekeeping. The pinned 1.19.8 cannot reach a current Docker daemon at all: docker-java 3.4.x negotiates API 1.32, Docker 29 requires 1.44 and refuses, so every Testcontainers test in this repo failed before starting a container. 2.0.5 carries docker-java 3.7.1 and connects. Bumping within 1.x does not help — I tried 1.21.3 first.
junit-jupiterandmongodbare dropped: declared but never used — nothing here imports@Testcontainers,@ContainerorMongoDBContainer— and 2.x does not publish them. Only core is needed, forComposeContainerandWait.One behavioural note for CI: 2.x removed containerised compose, so
ComposeContainershells out to thedockerCLI. Present on any GitHub runner; worth knowing if the build ever moves into a container that lacks it.Verification changes
InclusionProofused to carry both answers the aggregator can give — a certified leaf, and the absence of one — so every field was nullable and every consumer re-established which case it held. That is now split:The wire form expresses both shapes, so decoding it stays where the layout lives:
decodeOrAbsentreturns the proof or null and rejects any partial combination;fromCborrefuses anything but a leaf.What falls out: the whole status taxonomy for absent fields is gone —
MISSING_CERTIFICATION_DATA,MISSING_REFERENCE_TIME,INCOMPLETE_INCLUSION_PROOF,INCLUSION_CERTIFICATE_MISSING— because none of them can occur. Both certified-transaction decoders drop their guards. The poll loop branches on the response having no proof rather than on a status that meant the same thing, and itsswitchcollapses to anif. There is no null handling left anywhere in the verification path.The wire bytes do not change, and the interop test proves it rather than asserting it: a token minted by the published TypeScript SDK, which does not have this split, still decodes and verifies here unchanged.
The remaining behaviour changes:
REFERENCE_TIME_AFTER_ROUND). Read the comment on that check before relying on it — the bound is one-sided and does not stop back-dating. Filed as Inclusion proofs do not let a verifier detect a back-dated reference time, so expiresAt is unenforceable against a dishonest aggregator aggregator-go#186.expiresAtis validated where it is accepted rather than failing later inside CBOR encoding.One thing with no TypeScript counterpart
Both deadline comparisons are unsigned. TypeScript's
bigintdoes not wrap; a Javalongdoes.CborDeserializer.CborUnsignedLong.asLongsays so in its own javadoc: a CBOR unsigned integer at or above 2^63 comes back as a negative long. A signed comparison would read such a reference time as earlier than every deadline and wave an expired request straight through — while the leaf value, computed from the same bits, still verified. BothREQUEST_EXPIREDand the round bound now useLong.compareUnsigned.This was found by review, not by a failing test, and it would have shipped otherwise.
Interop tests — the part that would have caught this
Each SDK builds a token from entirely fixed inputs (keys, salt, token type, state mask, deadline, round clock; RFC 6979 signing on both sides), commits it, and decodes and fully verifies the other's.
The argument for them, concretely: revert
Token.VERSIONto 1 — the state this branch starts from —The existing golden vectors are structurally incapable of seeing a container divergence, because
CertificationDatabytes never move.The two SDKs'
UnicityCertificatetest fixtures differ in three padding fields, so a shared token hex vector is impossible. It is not needed: each side reads the producer's certificate and trust base out of the fixture bundle.TestAggregatorClientgains a reference-time setter so a generated vector is byte-reproducible.The TypeScript half is unicitynetwork/state-transition-sdk-js#149.
Integration tests — unverified, please read
AggregatorStackstarts the compose stack from the TypeScript SDK's own file — BFT root node, mongodb, redis, pinned aggregator build — waits for consensus to certify a round rather than for the healthcheck, and tears it down.RequestDeadlineIntegrationTestmirrors the TypeScript cases.Tagged
integration, so the existingintegrationTesttask picks it up andtestkeeps excluding it. No build change was needed.These 8 tests have not been observed to pass. They compile, and I confirmed
testexcludes them, but the machine they were written on runs Docker 29, whose minimum API version (1.44) docker-java does not meet — it negotiates 1.32. I tried bumping Testcontainers to 1.21.3 and it did not help, so I reverted that rather than ship a change that fixes nothing. Please run./gradlew integrationTeston a normal Docker host before trusting them.Worth knowing separately: Testcontainers 1.19.8 as pinned here cannot talk to Docker 29 at all, so anyone on a current daemon cannot run integration tests in this repo regardless of this change.
Verification
./gradlew build— 142 tests, 0 failures (the branch base had 140; the split removed cases that are now unrepresentable and added decode-boundary ones), checkstyle clean. Run in agradle:8.5-jdk21container; there is no JDK on the machine this was written on../gradlew integrationTest— 9 tests, 0 failures, 0 skips: the 8 deadline cases plus the interop mint, all against the real aggregator the stack starts.Both token shapes carry the 3.0.0 container prefix
d99880 83 02 82 d99881 88 02 03.